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Abstract 



o 

— . , Let $ be a uniformly distributed random fc-SAT formula with n variables and m clauses. Non-rigorous sta- 

tistical mechanics ideas have inspired a message passing algorithm called Belief propagation guided decimation 
for finding satisfying assignments of €>. This algorithm can be viewed as an attempt at implementing a certain 
thought experiment that we call the decimation process. In this paper we identify a variety of phase transitions 
in the decimation process and link these phase transitions to the performance of the algorithm. 
Key words: random structures, phase transitions, fc-SAT, Belief Propagation. 



o 

^ ■ 1 Introduction 

Let fc > 3 and n > 1 be integers, let r > be a real, and set m = [to,] . Let <J? = «l?fc(n, m) be a propositional 
formula obtained by choosing a set of m clauses of length fc over the variables V = {x%, . . . , x n } uniformly at 
random. For k, r fixed we say that $ has some property V with high probability ('w.h.p.') if linin^oo P [*& 6 V] = 
1. 

The interest in random fc-SAT originates from the experimental observation that for certain densities r the 
random formula <J> is satisfiable w.h.p. while a large class of algorithms, including and particularly the workhorses 
of practical SAT solving such as sophisticated DPLL-based solvers, fail to find a satisfying assignment effi- 
ciently JIT) . Over the past decade, a fundamentally new class of algorithms has been proposed on the basis 
of ideas from statistical physics J6][l6]. Experiments performed for k = 3, 4, 5 indicate that these new 'message 
^vq | passing algorithms', namely Belief Propagation guided decimation and Survey Propagation guided decimation 

('BP/SP decimation'), excel on random fc-SAT instances iTHl . Indeed, the experiments indicate that BP/SP dec- 
imation find satisfying assignments for r close to the threshold where 3? becomes unsatisfiable w.h.p. Generally, 
SP decimation is deemed conceptually superior to BP decimation. 

For example, in the case fc = 4 the threshold for the existence of satisfying assignments is conjectured to be 
m/n ~ r4 « 9.93 11511 . According to experiments from 0131 . SP decimation finds satisfying assignments for 
densities up to r = 9.73. Experiments from lfT9l suggest that the "vanilla" version of BP decimation succeeds 
up to r = 9.05. Another version of BP decimation (with a different decimation strategy from @) succeeds up to 
r = 9.24, again according to experimental data from ifPJl . By comparison, the currently best rigorously analyzed 
algorithm is efficient up to r = 5.54 IfTDl . while zChaf f , a prominent practical SAT solver, becomes ineffective 
beyond r = 5.35 lfl3l . 

Since random fc-SAT instances have widely been deemed extremely challenging benchmarks, the stellar exper- 
imental performance of the physicists' message passing algorithms has stirred considerable excitement. However, 
the statistical mechanics ideas that BP/SP decimation are based on are highly non-rigorous, and thus a rigorous 
analysis of these message passing algorithms is an important but challenging open problem. A first step was 
made in O, where it was shown that BP decimation does not outperform far simpler combinatorial algorithms 
for sufficiently large clause lengths fc. More precisely, the main result of ]8] is that there is a constant po > 
(independent of fc) such that the 'vanilla' version of BP decimation fails to find satisfying assignments w.h.p. if 
r > po2 fc /fc- By comparison, non-constructive arguments show that w.h.p. $ is satisfiable if r < = 2 k In 2 — fc, 
and unsatisfiable if r > 2 k In 2 (5J5). This means that for fc 3> po sufficiently large, BP decimation fails to find 
satisfying assignments w.h.p. already for densities a factor of (almost) fc below the threshold for satisfiability. 
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Experiment 2.1 ('decimation process') Input: A satisfiable fc-CNF $. 

Result: A satisfying assignment a : V — > {0, 1} (with 0/1 representing 'false'/'true'). 

0. Let $o = $. 

1 . For t = 1, . . . , n do 

2. Compute the fraction M xt (® t -i) of all satisfying assignments of in which the 
variable x< takes the value 1. 

3. Assign a(x t ) = 1 with probability M xt ($t-i), and let a(x t ) = otherwise. 

4. Obtain the formula $ t from $ t _i by substituting the value cr(a; t ) for x t and simplifying 
(i.e., delete all clauses that got satisfied by assigning x t , and omit x t from all other 
clauses). 

5. Return the assignment a. 

Figure 1: The decimation process. 

The analysis performed in |8] is based on an intricate method for directly tracking the execution of BP deci- 
mation. Unfortunately this argument does little to illuminate the conceptual reasons for the algorithms' demise. 
In particular, [8 1 does not provide a link to the statistical mechanics ideas that inspired the algorithm. The present 
paper aims to remedy these defects. Here we study the decimation process, an idealized thought experiment that 
the BP decimation algorithm aims to implement. We show that this experiment undergoes a variety of phase tran- 
sitions that explain the failure of BP decimation for densities r > po ■ 2 k /k. Our results identify phase transitions 
jointly in terms of the clause/variable density r and with respect to the time parameter of the decimation process. 
The latter dimension was ignored in the original statistical mechanics work on BP |6] \W\ but turns out to have a 
crucial impact on the performance of the algorithm. On a non-rigorous basis, this has been pointed out recently by 
Ricci-Tersenghi and Semerjian |fl9l , and our results can be viewed as providing a rigorous version of (substantial 
parts of) their main results. The results of this paper can also be seen as a generalization of the ones obtained in JTJ 
for random fc-SAT, and indeed our proofs build upon the techniques developed in that paper. 

2 Results 

BP decimation is a polynomial-time algorithm that aims to (heuristically) implement the 'thought experiment' 
shown in Fig. Q] IfTSl [T91 , which we call the decimation process^ A moment's reflection reveals that, given 
a satisfiable input formula <£>, the decimation process outputs a uniform sample from the set of all satisfying 
assignments of $. The obvious obstacle to actually implementing this experiment is the computation of the 
marginal probability M Xt ($ t -i) that x t takes the value 'true' in a random satisfying assignment of $t_i, a #P- 
hard problem in the worst case. Yet the key hypothesis underlying BP decimation is that these marginals can 
be computed efficiently on random formulas by means of a message passing algorithm. We will return to the 
discussion of BP decimation and its connection to Experiment s. 1 I below. 

We are going to study the decimation process when applied to a random formula <J? for densities r < 2 k In 2 — k, 
i.e., in the regime where # is satisfiable w.h.p. More precisely, conditioning on # being satisfiable, we let & t be 
the (random) formula obtained after running the first t iterations of Experiment 12.11 The variable set of this 
formula is V t = {xt+i, ■ ■ ■ ,x n }, and each clause of <fr t consists of at most k literals. Let 5(<&t) C {0, l} Vt 
be the set of all satisfying assignments of & t - We say that almost all a 6 S($>t) have a certain property A if 
L4n«S(* t )| = (l-o(l))|5(* t )|. 

We will identify various phase transition that the formulas 3> t undergo as t grows from 1 to n. As it turns out, 
these can be characterized via two simple parameters. The first one is the clauses density r <~ m/n. Actually, it 
will be most convenient to work in terms of 

p = kr/2\ 

sothatm/n ~ p-2 k /k. We will be interested in the regime po < p < k In 2, where po is aconstant (independent of 
k). The upper bound k In 2 marks the point where satisfying assignments cease to exist 101 . The second parameter 

1 Several different versions of BP decimation have been suggested. In this paper we refer to the simplest but arguably most natural one, also 
considered in |8 18 19']. Other versions decimate the variables in a different order, allowing for slightly better experimental results |6 13 |. 



is the fraction 

6 = 1- t/n 

of 'free' variables (i.e., variables not yet assigned by time t). 

The symmetric phase. Let $ be a fc-CNF on V, let 1 < t < n, let <& t be the formula obtained after t steps of 
the decimation process, and suppose that a G 5($t). A variable x G Vt is loose if there is t G <S($ t ) such that 
a(x) ^ t(x) and d(cr, r) < Inn, where <i(-, •) denotes the Hamming distance. For any x G Vt we let 

= lfreS(* t ):<r(*) = l}| 
xl t; |5($ t )l 

be the marginal probability that x takes the value 'true' in a random satisfying assignment of $ t . 
Theorem 2.2 There are constants k$, po > smc/i that for k > kg, po < p < khi2 — 2\nk, and 

[ / lnlnp 10 N 
fc • > exp p H + - 

L V P 

the random formula <fr t /ias the following properties w.h.p. 

1. In almost all satisfying assignments a G <S(3>t) flf Zeasf 0.99#n variables are loose. 

2. At least 9n/3 variables x E V t satisfy M x (& t ) G [0.01, 0.99]. 

3. The average distance of two random satisfying assignments satisfies 

d(a,T)/\S(& t )\ 2 > 0.496>n. 

<T,re5(* t ) 

Intuitively, Theorem l2.2l can be summarized as follows. In the early stages of the decimation process (while 
9 is 'big'), most variables in a typical a G 5(<&t) are loose. Hence, the correlations amongst the variables are 
mostly local: if we 'flip' one variable in a, then we can 'repair' the unsatisfied clauses that this may cause by 
simply flipping another Inn variables. Furthermore, for at least a good fraction of the variables, the marginals 
M x (&t) are bounded away from 0/1. Finally, as the average distance between satisfying assignments is large on 
average, the set <S(<&t) is 'well spread' over the Hamming cube {0, l} Vt . 

Shattering and rigidity. Let <I> be a fc-CNF and let a G <S($t). For an integer u> > 1 we call a variable x € Vt 
uj-rigid if any t G <S($t) with cr(x) ^ t(x) satisfies d(a, r) > ui. 

Furthermore, we say that a set S G {0, l} Vt is (a, (3)-shattered if it admits a decomposition S = Ui=i 
into pairwise disjoint subsets such that the following two conditions are satisfied. 

SHI. We have \R t \ < exp(-a6n)\S\ for all 1 < i < N. 

SH2. If 1 < i < j < N and a G R t , r G Rj, then dist(<r, r) > (39n. 

Theorem 2.3 There are constants ko, po > such that for k > ko, Po < p < fc In 2 — 2 In fc, 



— (l + 2p~ 2 ) <fc6»<cxp 
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pi 
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f/ie random formula <I? t /ias the following properties w.h.p. 

1. In almost all a G 5(<&t) of Zeasf 0.99#n variables are fl(n)-rigid. 

2. There exist a = a(k, p) > 0, /3 = /3(fc, p) > 5mc/i f/iaf <S(4?t) is fi)-shattered. 

3. At least 9n/3 variables x <E V t satisfy M x ($ t ) G [0.01, 0.99]. 

4. The average distance of two random satisfying assignments is at least 0A99n. 



Thus, if the fraction of free variables lies in the regime (Q3, then in most satisfying a 6 <S($f) the values 
assigned to 99% of the variables are linked via long-range correlations: to 'repair' the damage done by flipping a 
single rigid variable it is inevitable to reassign a constant fraction of all variables. This is mirrored in the geometry 
of the set <S($t): it decomposes into exponentially many exponentially tiny subsets, which are mutually separated 
by a linear Hamming distance ft(n). Yet as in the symmetric phase, the marginals of a good fraction of the 
free variables remain bounded away from 0/1, and the set <S(4?t) remains 'well spread' over the Hamming cube 

{o,i} Vt . 

The condensation phase. Let a > 0. We say that a set S C {0, 1} n is a-condensed if for any a, t <G S we have 
dist(<7, t) < an. 

Theorem 2.4 There are constants fco , po > such that for k > ko, po < p < fc In 2 — 2 In fc, ancf 

In p < fc ■ 6» < (1 - p- 2 ) ■ pi (In 2) (2) 
f/ie random formula <l> t /ias the following properties w.h.p. 

1. In almost all a £ 5(<&t) flf /easf O.990n variables are fl(n)-rigid. 

2. The set 5(3>t) i's exp(2 — p) / k-condensed. 

3. At least 0.998n variables x <E V t satisfy M x (& t ) £ [0, 2~ k / 2 ] U [l - 2" fc / 2 , l]. 

4. There is a set R C Vt of size \R\ > 0.999n such that for any o~,t £ <S($t) /iflve 

\{x e R: <t(x) ^ r(x)}| < k2~ k n. 

In other words, as the decimation process progresses to a point that the fraction 6 of free variables satisfies (O, 
the set of satisfying assignments shrinks into a condensed subset of {0, l} Vt of tiny diameter, in contrast to a 
well-spread shattered set as in Theorem I2.3I Furthermore, most marginals M x (& t ) are either extremely close to 
or extremely close to 1. In fact, there is a large set R of variables on which all satisfying assignments virtually 
agree (more precisely: any two can't disagree on more than k2~ k n variables in R). 

The forced phase. We call a variable x forced in the formula $ t if $ t has a clause that only contains the variable 
x (a 'unit clause'). Clearly, in any satisfying assignment x must be assigned so as to satisfy this clause. 

Theorem 2.5 There are constants ko, po > such that for k > ko, po < p < k In 2 — 2 In k, and 

l/n<k-e<]n(p)(l-10/]np) (3) 

the random formula <l> t has the following properties w.h.p. 

1. At least 0.999n variables are forced. 

2. The set 5($t) is exp(2 — p) / k-condensed. 

Belief Propagation. As mentioned earlier, the BP decimation algorithm is an attempt at implementing the dec- 
imation process by means of an efficient algorithm. The key issue with this is the computation of the marginals 
M Xt ($t-i) m step 2 of the decimation process. Indeed, the problem of computing these marginals is #P- 
hard in the worst case. Thus, instead of working with the 'true' marginals, BP decimation uses certain numbers 
p Xt ($t_i , to) that can be computed efficiently, where uj > 1 is an integer parameter. The precise definition of the 
p Xt ($t-i, w) can be found in Appendix lAl (or J6)). Basically, they are the result of a 'local' dynamic program- 
ming algorithm ('Belief Propagation') that depends upon the assumption of a certain correlation decay property. 
For given fc, p, the key hypothesis underpinning the BP decimation algorithm is 

Hypothesis 2.6 For any e > there is uj = w(e, k,p, n) > 1 such that w.h.p. for all 1 < t < n we have 
1^ ($*_!, w) - M xt (<f> t -x)\ < £• 

In other words, Hvpothesis l2.6l states that throughout the decimation process, the 'BP marginals' p Xt ($t_i , u>) 
are a good approximation to the true marginals M Xt ($ f _i). 



Theorem 2.7 There exist constants Co, fco,Po > such that for all k > ko, an d pa < P < fcln2 — 21nfc the 
following is true for any integer u = uj(k, p, n) > 1. Suppose that 

colnO) < k-6 <p/hx2. (4) 

Then for at least 0.999n variables x £ Vt we have p x (^ t T^>) S [0.49, 0.51] . 

The proof is based on the techniques developed in J8); the details are omitted from this extended abstract^ 
Comparing Theorem 12 . 4 1 with Theorem 12.71 we see that w.h.p. for 8 satisfying (0| most of the 'true' marginals 
M x ($i) ar e very close to either or 1, whereas the 'BP marginals' lie in [0.49,0.51]. Thus, in the regime 
described by (O the BP marginals do not provide a good approximation to the actual marginals. 

Corollary 2.8 There exist constants cq, fco, po > such that for all k > ko, po < p < k In 2 — 3 In k Hypothe- 
sis \2.6\ is untrue. 

Summary and discussion. Fix k > ko and p > po- Theorems 12 . 2\]2 . 5 1 show how the space of satisfying assign- 
ments of 3> t evolves as the decimation process progresses. In the symmetric phase k9 > exp((l + o p (l))p) where 
there still is a large number of free variables, the correlations amongst the free variables are purely local ('loose 
variables'). As the number of free variables enters the regime (1 + o p (l))p/ln2 < k6 < cxp((l — o p (l))p), 
the set iS(€>i) of satisfying assignments shatters into exponentially many tiny 'clusters', each of which comprises 
only an exponentially small fraction of all satisfying assignments. Most satisfying assignments exhibit long-range 
correlations amongst the possible values that can be assigned to the individual variables ('rigid variables'). This 
phenomenon goes by the name of dynamic replica symmetry breaking in statistical mechanics lfl4l . 

While in the previous phases the set of satisfying assignments is scattered all over the Hamming cube (as 
witnessed by the average Hamming distance of two satisfying assignments), in the condensation phase (1 — 
o p (l))lnp < k6 < (1 — o p (l))p/ln2 the set of satisfying assignments has a tiny diameter. This is mirrored 
by the fact that the marginals of most variables are extremely close to either or 1. Furthermore, in (most of) 
this phase the estimates of the marginals resulting from Belief Propagation are off (Theorem 12.7) . As part 4 of 
Theorem l2.4l shows, the mistaken estimates of the Belief Propagation computation would make it impossible for 
BP decimation to penetrate the condensation phase. More precisely, even if BP decimation would emulate the 
decimation process perfectly up until the condensation phase commences, with probability 1 — cxp(— n(n)) BP 
decimation would then assign at least k2~ k n variables in the set R from part 4 of Theorem 12.41 'wrongly' (i.e., 
differently than they are assigned in any satisfying assignment). In effect, BP decimation would fail to find a 
satisfying assignment, regardless of its subsequent decisions. Finally, in the forced phase kO < (1 — o p (l)) In p 
there is an abundance of unit clauses that make it easy to read off the values of most variables. However, getting 
stuck in the condensation phase, BP decimation won't reach this regime. 

These results suggest that the reason for the failure of BP decimation is the existence of the condensation phase. 
Intuitively, in the condensation phase the marginals are governed by genuinely global phenomena (essentially 
expansion properties) that elude the inherently local BP computation. By contrast, it is conceivable that BP does 
indeed yield the correct marginals in the previous phases. Verifying or falsifying this remains an important open 
problem. 

3 Related work 

The statistical mechanics perspective. BP/SP decimation are inspired by a generic but highly non-rigorous 
analysis technique from statistical mechanics called the cavity method |6). This technique is primarily destined 
for the analysis of phase transitions. It is based on the (unproven) replica symmetry breaking hypothesis, which 
aims to characterize the possible types of correlations amongst the variables fl4l . 

In ||6][T4) the cavity method was used to study the structure of the set 5(3?) of satisfying assignments (or, more 
accurately, properties of the Gibbs measure) of the undecimated random formula <I>. Thus, the results obtained 
in that (non-rigorous) work identify phase transitions solely in terms of the formula density p. On the basis of 
these results, it was hypothesized that (certain versions of) BP decimation should find satisfying assignments up 
to p ~ In k or even up to p ~ fc In 2 lfl4l . The argument given for the latter scenario in lfl"4l is that the key obstacle 
for BP to approximate the true marginals is condensation. In terms of the parameter p, the condensation threshold 
was (non-rigorously) estimated to occur at p = k In 2 — 3k2~ k ~ 1 In 2. However, |8J shows that (the basic version 
of) BP decimation fails to find satisfying assignments already for p > po, with po a constant independent of k. 

The explanation for this discrepancy is that |6][14] neglect the time parameter 6 = 1 — t/n of the decimation 
process. As Theorem 12 . 4 1 show s . even for fixed p > po (independent of k) condensation occurs as the decimation 



2 In the appendix we indicate how Theorem l2.7l follows from the results of (8J. 



process proceeds to 6 in the regime ©. This means that decimating variables has a similar effect on the geometry 
of the set of satisfying assignments as increasing the clause/variable density. On a non-rigorous basis an analysis 
both in terms of the formula density p and the time parameter 8 was carried out in |[T9l . Thus, our results can 
be viewed as a rigorous version of |[T9l (with proofs based on completely different techniques). In addition, 
Theorem l2.7l confirms rigorously that for p, 8 in the condensation phase, BP does not yield the correct marginals. 

The present results have no immediate bearing on the conceptually more sophisticated SP decimation algo- 
rithm. However, we conjecture that SP undergoes a similar sequence of phase transitions and that the algorithm 
will not find satisfying assignments for densities p > po, with po a certain constant independent of fc. 
Rigorous work. Theorem 12. 3 1 can be viewed as a generalization of the results on random fc-SAT obtained in 0] 
(which additionally deals with further problems such as random graph/hypergraph coloring). In JT] we rigorously 
proved a substantial part of the results hypothesized in [;14|| on shattering and rigidity in terms of the clause/variable 
density p; this improved prior work |]2] |5] H]. The new aspect of the present work is that we identify not only a 
transition for shattering/rigidity, but also for condensation and forcing in terms of both the density p and the time 
parameter 8 of the decimation process. As explained in the previous paragraph, the time parameter is crucial to 
link these phase transitions to the performance of algorithms such as BP decimation. 

In particular, from Theorem 12.31 we can recover the main result of [T| on random fc-SAT. Namely, if p > 
In fc + 2 In In fc + 2, then (fTJ is satisfied even for 6 = 1, i.e., the undecimated random formula 4? has the properties 
1.-4. stated in Theorem 12. 3 1 Technically, the present paper builds upon the methods developed in ifT). In addition, 
new arguments are needed to accommodate the time parameter 6, to prove the statements on the marginals of the 
variables in Theorems 12 .2112 .41 and to establish the condensation phenomenon (Theorem l2.4l >. 

The best current rigorous algorithmic results for random fc-SAT are J7] [10] [TT] Q2] • For general k, the best 
current algorithm succeeds up to p ~ In fc 171 . 

4 Analyzing the decimation process 

In the rest of the paper, we are going to sketch the proofs of the main results^ In this section we perform some 
groundwork to facilitate a rigorous analysis of the decimation process. The key problem is to get a handle on the 
following experiment: 

Dl. Generate a random formula <I>, conditioned on <& being satisfiable. 
D2. Run the decimation process for t steps to obtain <fr t . 
D3. Choose a satisfying assignment cr t G 5(<&t) uniformly at random. 
D4. The result is the pair (<& t , cr t ). 

As throughout the paper we only work with densities m/n where <& is satisfiable w.h.p., the conditioning in 
step Dl is essentially void. Recalling that the outcome of the decimation process is a uniformly random satisfying 
assignment of €>, we see that the following experiment is equivalent to D1-D4: 

Ul. Generate a random formula €>, conditioned on <& being satisfiable. 

U2. Choose cr G <S($) uniformly at random. 

U3. Substitute cr(xi) for Xi for 1 < i < t and simplify to obtain a formula $ t . 
U4. The result is the pair (3> t , er t ), where <r t : V t — > {0, 1} , x h-> cr(x). 

Fact 4.1 The two probability distributions induced on formula/assignment pairs by the two experiments D1-D4 
and U1-U4 are identical. 

Still, an analysis of U1-U4 seems difficult because of U2: it is unclear how to analyze (or implement) this step 
directly. Following [ 1 1, we will surmount this problem by considering yet another experiment. 

PI. Choose an assignment cr' G {0, 1} V uniformly at random. 

P2. Choose a formula <!>' with m clauses that is satisfied by cr' uniformly at random. 

3 Full proofs can be found in the appendix. 



P3. Substitute cr'(xi) for X{ for 1 < i < t and simplify to obtain a formula 3? J. 
P4. The result is the pair cr' t ), where a' t : V t -> {0, 1}, ii-> <x'(x). 

The experiment P1-P4 is easy to implement and, in effect, also amenable to a rigorous analysis. For given the 
assignment er', there are (2 fe — 1) (?) clauses in total that evaluate to 'true' under er', and to generate €>' we merely 
choose m out of these uniformly and independently. Unfortunately, it is not true that the experiment P1-P4 is 
equivalent to U1-U4. However, we will employ a result from JT] that establishes a connection between these two 
experiments that is strong enough to extend many results from P1-P4 to U1-U4. 

To state this result, observe that P1-P4 and U1-U4 essentially only differ in their first two steps. Thus, let 
Afc(n, to) denote the set of all pairs (<1>, cr), where $ is a k-CNF on V = {xi, . . . , x n } with m clauses, and 
a G <5($). Let Uk(n, m) denote the probability distribution induced on Afc(n, m) by U1-U2, and let Vk{n, m) 
signify the distribution induced by P1-P2; this distribution is sometimes called the planted model. 

Theorem 4.2 (Ql) Suppose k > 4 and < p < fcm2 - k 2 /2 k . Let £ C A fe (n,m). If P Vk{n , m) [£} > 
1 - exp(-pn/2 k ) then P Uk (n,m) [£] = 1 - o(l). 



5 Shattering, pairwise distances, and condensation 

To prove shattering and condensation, we adapt arguments from Q] [2] [9] to the situation where we have the 
two parameters 6, p (rather than just p). Let (<fr t , er t ) be the (random) outcome of the experiment U1-U4. For 
< a < 1 let X Q ($ t ,cr t ) denote the number of satisfying assignments r G S(<f?t) with Hamming distance 
d(er t , t) = aQn. To establish the 'shattering' part of Theorem 12. 3 1 we are going to prove the following 



Claim 5.1 Under the assumptions of Theorem \2.3\ there exist ai < ai < 0.49, 0,3 > depending only on k, p 
such that w.h.p. we have 

X a (<&t,(Tt) — for all ai < a < C12, and (5) 
max I a ($(,(Ti) < exp(-a 3 ri) • |S(* t )| . (6) 

a<0.49 

Claim I5TI implies that for the outcome 3> t of the first t steps of the decimation process the set <S(<&t) shat- 
ters w.h.p. For by Fact 14. 1 1 Claim I5TI implies that w.h.p. almost all a t G S($t) are sucn that ((5J and (|6]l hold. 
Choose any such o~t,\ G S(&t) an d let Ri = {t G <S(3>t) : cr^i) < ain}. Then, choose cr t ,2 G 5(<&t) \ i?i 
satisfying (O and ©, let ^ = {t £ <S($t) \ R\ : d(r, (74,2) < and proceed inductively until all remaining 

satisfying assignments violate either (|5]l or ©. Let R\, . . . , Rm be the classes constructed in this way and let 
Ro = S(&t) \ Uili Ri- ^ n additional (simple) argument is needed to show that |i?o| < exp(— Q.(n))\S(&t)\ 
w.h.p. The decomposition R , . . . , Rn witnesses that 5(€> t ) shatters. 

With respect to pairwise distances of satisfying assignments, (0 implies that w.h.p. only an exponentially small 
fraction of all satisfying assignments of $ t lies within distance < 0A98n of er t . It is not difficult to derive the 
statement made in Theorem 12. 3 1 on the average pairwise distance from this. In addition, the fact that the average 
pairwise distance of satisfying assignments is > 0.496*?? w.h.p. implies in combination with a double counting 
argument the claim about the marginals M x ($t) in Theorems 12 . 21 and |2 . 3 1 

To establish Claim I5TI we will work with the experiment P1-P4 and use Theorem 14.21 to transfer the result 
to the experiment U1-U4. Thus, let (&' t ,cr' t ) be the (random) outcome of experiment P1-P4, and assume that 
k, p,0 are as in Theorem 12. 3 1 To prove (|5) we need to bound X a ($>' t ,a' t ) from above, for which we use the 'first 
moment method'. Indeed, by standard arguments (similar to those used in O) the expectation of X a (<&' t , er' t ) 
satisfies i ]nEX a (<&' t , a[) < i>{a), with 

iP(a) = -aBlna- (1 - a)01n(l - a) + ^ In ( 1 - 1 ~ (1 ~ a6 ^ 



k \ 2 k - 1 

Thus, in order to prove that max ai<Q<a2 X a (&' t ,o-' t ) = w.h.p. we would just have to prove that max ai<Q<a2 ip(a) < 
(so that Markov's inequality implies that X a = w.h.p.). But as our goal is to prove a result about the 
X a (<f>t, ft) 0-e., the experiment U1-U4), we need to prove a slightly stronger bound, namely 



max -0(a) < ~p/2 k - 

a\ <a<a 2 



(7) 



Then Markov's inequality and Theorem 14.21 imply the first part of Claim [57X1 Via elementary calculus, one can 
show that (Q) holds with ai = cxp(2 — p) — e and a-x — exp(2 — p) + e for a sufficiently small e > 0. 

To prove © we bound EX a from above by a similar first moment argument. But in addition, we need a lower 
bound on |<S(3> 4 )|. To derive this lower bound, we need 

Theorem 5.2 (0) Assume k >Aandp< fcln2- k 2 /2 k . Thenw.h.p. ±ln|.S(*)| > ln2 + 2 fc fln(l - 2~ k ) - 

0. 99p/2 k . 

In combination with a double counting argument, Theorem 15.21 implies the following lower bound on <S($ t ), 
which entails the second part of Claim IBTTI 

Corollary 5.3 Let (*t,er t ) be the outcome o/Ul-U4. Let 1 < t < n. Then w.h.p. -ln|<S(* t )| > 6>ln2 + 

2 k § ln(l-2- fe )-#- 

The proof of the 'condensation' part of Theorem 12. 4 1 is based on similar arguments. Basically, to show con- 
densation we need to prove that max Q>04 X a (§j, at) < with probability 1 — exp(— f2(n)), where we let 
04 = cxp(2 — p). This is done via the first moment method and boils down to proving that ip(a) < —p/2 k for all 
a > ai. 

6 Rigid variables 

Assume that k, p, 9 satisfy the assumptions of Theorem 12.31 Let (<I> t , cr t ) be the (random) outcome of U1-U4. 
Our goal is to show that w.h.p. most variables x G Vt are rigid. 

What is the basic obstacle that makes it difficult to 'flip' the value of xl Observe that we can simply assign x 
the opposite value 1 — cr t (x), unless <fr t has a clause C in which either x or x is the only literal that is true under 
cr t . If there is such a clause, we say that x supports C. But even if x supports a clause C it might be easy to flip. 
For instance, if C features some variable y ^ x that does not support a clause, then we could just flip both x, y 
simultaneously. Thus, to establish the existence of f2(n)-rigid variables we need to analyze the distribution of the 
number of clauses that a variable supports, the probability that these clauses only consists of variables that support 
further clauses, the probability that the same is true of those clauses, etc. 

This analysis can be performed fairly neatly for the outcome ($' t , er' t ) of the experiment P1-P4. Let us sketch 
how this works, and why rigidity occurs at k9 = cxp((l + o(l))p) (cf. ((TJ). For a variable x £ Vt we let S x be the 
number of clauses supported by x. Given the assignment er' chosen in step PI, there are a total of (?Zi) possible 
clauses that x supports. Since in step P2 we include m out of the (2 k — 1) (?) possible clauses satisfied under a' 
uniformly and independently, we get 

-»(;:;) ^--D^y'^m^-^^,. 

In fact, S x is binomially distributed. Hence, P [S x = 0] < exp(— p). Thus, the expected number of variables 
x G Vt with S x = is < #nexp(— p). Furthermore, if we condition on S x = j > 1, then the actual clauses 
C\, . . . , Cj supported by x are just independently uniformly distributed over the set of all ClZi) possible clauses 
that x supports. Therefore, the expected number of variables y G Vt with S y = occurring in one of these clauses 
Cj is (1 + o(l))(fc — 1) • 6>cxp(— p) < k9exp(—p). Hence, if 9 is as in ((TJ, then this number is < exp(— 2)/p, 

1. e., 'small' for p > pa sufficiently big. Thus, we would expect that most clauses supported by x indeed consist 
exclusively of variables that support other clauses. This heuristic argument shows that for 9 as in ([T} we can 
plausibly expect most variables to be rigid. 

Let us now indicate how this argument can be carried out in detail. Analyzing the distribution of the variables 
S x in the experiment P1-P4 and extending the result to the experiment U1-U4 via Theorem 14.21 and setting 
C = p 2 / exp(p), we obtain the following. 

Proposition 6.1 Suppose that k,p,9 satisfy the assumptions of Theorem 12.31 Then w.h.p. in a random pair 
er t ) generated by the experiment U1-U4 no more than 2C,9n variables in Vt support fewer than three clauses, 

To establish rigidity, we need to show that most variables support clauses in which only variables occur that 
support other clauses. To express this, we say that S C Vt is t-self-contained if each x G S supports at least two 
clauses of $ 4 that contain variables from S only. From Proposition |6T| we can derive the following. 



Proposition 6.2 Suppose that k,p,9 satisfy the assumptions of Theorem \2.3\ The outcome (3> t ,er t ) o/Ul-U4 
has a t-self-contained set of size (1 — ZCjOn w.h.p. 

Suppose that (3> f , cr t ) has a self-contained set S of size (1 — 3()6n. To flip the value of a variable x S S 
we need to also flip one other variable from each of the (at least two) clauses that x supports and that consist of 
variables from S only. As each of these two variables, in turn, supports at least two clauses comprised of variables 
from 5* only, we need to also flip further variables in those. But these variables are again contained in S. This 
suggests that attempting to flip x will entail an avalanche of further flips. Indeed, the expansion properties of the 
random formula <& 4 imply the following. 

Proposition 6.3 Suppose that k,p,6 satisfy the assumptions of Theorem \2.3\ There is \ = x(k,p) > such 
that the outcome ($ t ,<r t ) o/Ul-U4 has the following property w.h.p.: all variables that are contained in a 
t-self-contained set are xn-rigid. 

Propositions 16.21 and 16 . 3 1 directly imply part 1 of Theorem 12.3 1 Self-contained sets also play a key role in the 
proof of Theorem l2.4l Propositions 16. 2l and l6. 3 l ean be extended to the regime of 9 as in Theorem l2.4l and the set 
R in part 4 of that theorem is simply a t-self-contained set. Expansion properties of the random formula together 
with the bound on the diameter of the set S(&t) of satisfying assignments from part 2 of Theorem 12 . 4 1 imply that 
there are no two satisfying assignments that disagree on more than kn/2 k variables from R. In combination with 
a double-counting argument, this implies the statement on the marginals in part 3 of Theorem 12.41 Finally, the 
claim about forced variables in Theorem l2.5l can be proved via a similar (but simpler) argument as sketched in this 
section. 
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Appendix 



Appendix[A]contains a discussion of Belief Propagation. The remaining appendices contain the full proofs of the 
results stated in Section|2] AppendixlBl gives an overview of how the proofs are organized. 



A Detailed description of BP decimation 

The BP decimation algorithm can be viewed as an attempt at implementing the decimation process (Experi- 
ment l2.ll ). As mentioned earlier, the key issue with this is the computation (or approximation) of the the marginals 
AI x ($>t-i)- BP decimation basically tries to approximate these marginals by means of a 'local' computation. 

For clearly, the marginals M x (ffr t -i) are influenced by 'local' effects. For instance, if x occurs in a unit 
clause a of <3?i-i, i.e., a clause of length one, then x must be assigned so as to satisfy a. Hence, if x appears in 
a positively, then M x (<fr t -i) = L and otherwise M x (§ t -i) = 0. Similarly, if x occurs only positively in i>t_i, 
then M x (<fr t -i) > 1/2. More intricately, if x occurs in a clause a that contains another variable y that appears is 
a unit clause b, then this will affect the marginal of x. 

The key hypothesis underlying BP dec is that in random formulas such local effects determine the marginals 
M x ($t-i) asymptotically. To define 'local' precisely, we need a metric on the variables/clauses. This metric is 
the one induced by the factor graph G = G($t-i) of $t-i> which is a bipartite graph whose vertices are the 
variables Vt-i — {x t , . . . , x n } and the clauses of $t-i- Each clause is adjacent to the variables that occur in it. 
For an integer cj > 1 let ^(xt) signify the set of all vertices of G that have distance at most 2oj from xt- Then 
the induced subgraph G [N^fat)] corresponds to the sub-formula of &t-i obtained by removing all clauses and 
variables at distance more than 2lj from x t . Note that all vertices at distance precisely 2u> are variables, so that any 
satisfying assignment of $ induces a satisfying assignment of the sub-formula. Let us denote by M It ($ ( _i,w) 
the marginal probability that xt takes the value 1 in a random satisfying assignment of this sub-formula. 

Of course, for a worst-case formula $ the 'local' marginals M Xt (3>t-i) w) may be just as difficult to compute 
as the overall marginals M Xt ($t_i ) themselves. Therefore, BP decimation employs an efficient dynamic program- 
ming heuristic called Belief Propagation ('BP'), which yields certain values fi Xt ($ t _i, cj) G [0, 1]; we will state 
this heuristic below. If the induced subgraph G [N u (xt)] is a tree, then indeed p, Xi ($t-i, w) = M Xt w). 
Moreover, standard arguments show that in a random formula <& actually G \N u {xt)\ is a tree w.h.p. so long as 
uj = o(lnn). Of course, more generally, in order to obtain an efficient algorithm it would be sufficient for the 
BP outcomes /i, Xt ($t_i,o;) to approximate the true overall marginals M It (f ( _i) well for some polynomially 
computable and polynomially bounded function u> = uj(n) > 1. 

To define the numbers /i, Xt ($t_i,o;) formally, we need to define Belief Propagation for fc-SAT. To this end, 
let N(v) denote the neighborhood of a vertex v of the factor graph G($t_i). For a variable x G Vt and a 
clause a G N(x) we will denote the ordered pair (x, a) by x — > a. Similarly, a — > x stands for the pair (a, x). 
Furthermore, we let sign(a;, a) = 1 if x occurs in a positively, and sign(x, a) = — 1 otherwise. 

The message space M($ t _i) is the set of all tuples (A*x->-o(C))seVt, oeiV(a:),Ce{o,l} suc h that Ms-xjCC) S [0, 1] 
and (i x -m(Q) + n x ^ a (l) = 1 for all x, a, (. For fi G M($) we define n a ^x(() = 1 if C = (1 + sign(x, o))/2, 
and 

/>\ -, TT ( 1 - sign(y, a) \ 

yeN(a)\{x} V J 

otherwise. Furthermore, we define the belief propagation operator BP as follows: for any \i G M($ t _i) we 
define BP(/x) G M(<f> t -i) by letting 



n 



( BP (M))x->a(0 = — ^ — (9) 

i] n b ^ x (o)+ n /W 1 ) 

b£N(x)\{a} b<£N{x)\{a} 

unless the denominator equals zero, in which case (BP(/i)) x _ ) . a (f) = |- 

Finally, the values /j. x ($> t ,L}) are defined as follows. Let /n [0] = | • 1 G M($t_i) be the vector with all 



Algorithm A.l BPdec($) 

Input: A fc-CNF $ on V = {x\, ...,x n }. 
Output: An assignment a : V — > {0, 1}. 

0. Let $ = 

1 . For t= 1, . . . ,n do 

2. Use BP to compute ^(^t-i, w). 

3. Assign cr(a;t) = 1 with probability p, xt ($ t -i,u), and let <x(xt) = oth- 
erwise. 

4. Obtain the formula <&< from $ t _i by substituting the value a(x t ) for x t 
and simplifying. 

5. Return the assignment a. 

Figure 2: The BP decimation algorithm, 
entries equal to |. Moreover, define inductively \i [£] = BP(/_t [£ — 1]) for 1 < I < u. Then 

Y[ M6-^x(l) H 

^(*t-i,w) = — - (10) 

n /*&->»(o)M+ n M6-vx(i)M 

b£N(x) beN{x) 

for any a; G Vt> unless the denominator is zero, in which case we set /^(^t-i, oj) = |. 

The intuition here is that the p x ^ a (() are 'messages' from a variable x to the clauses a in which x occurs, 
indicating how likely x were to take the value f if clause a were removed from the formula. Based on these, (|8} 
yields messages p a ^x(C) from clauses a to variables x, indicating the probability that a is satisfied if x takes 
the value £ and all other variables y G iV(a) \ {x} are assigned independently to either value £ G {0, 1} with 
probability p y ^ a (£). The BP operator Q then uses these messages p a ^ x in order to 'update' the messages from 
variables to clauses. More precisely, for each x and a G N(x) the new messages (BP(/z)) a: ^. a (C) are computed 
under the hypothesis that all other clauses b G N(x) \ {a} are satisfied with probabilities fib^x(C) independently 
if x takes the value £. Finally, the difference between (O and ( TTOb is that the latter product runs over all clauses 
b G N(x). An inductive proof shows that, if for a variable x the subgraph G [N^(x)] of the factor graph is a tree, 
then in fact /j, x ($> t ,u}) = M x (&t,u>) J6|. Figure [2] shows the BP decimation algorithm. 



B Overview 

In Section|2]we described the main results of this paper arranged according to the various phases that the decima- 
tion process passes through. But to prove these results, it is necessary to proceed in a different order. To facilitate 
this, we will state the main results in the order in which the proofs proceed. We begin with the statements on the 
loose/rigid/forced variables. 

Theorem B.l There exist constants ko, po > such that for all k > kg and po < p < k In 2 — 2 In k the following 
three statements hold for a random pair (<& t , a t ) chosen from the experiment U1-U4 w.h.p. 

A Inlnp 10 

'V p p 



1. Ifke > exp p[l + ^£ + ^f) , then at least Q.999n variables x G V t are loose w.h.p. 

2. Ifl<k6< exp p i 1 — 3l p P J , then at least p 3 cxp(— p)9n variables x G Vt are Vt{n)-rigid w.h.p. 

3. If\a.(n)/n < 6 < (ln(p) — 10)/fc, then at least 0.99071 variables are forced w.h.p. 



The second type of statement concerns the global structure of the set of satisfying assignments, summarized 
in the following theorem. 

Theorem B.2 There exist constants ko, po > such that for all k > ko, and po < p < k In 2 — 2 In k the following 
three statements hold. 



1- If 

.V p p 



/-(l + p- 2 + 2 2 - fc )<fc0<cxp 
In 2 



then S(&t) is (exp(2 — p) — e, cxp(2 — p) + e)-shattered w.h.p. for some e = s(k, p) > 0. 

2. If 9 < (p — 1/p)/ (khi2), then S(<&t) is exp(2 — p)-condensed w.h.p. 

3. If 9 > p{\ + 2/p 2 )/(/cln2), then the average distance between two random elements of S{&t) is at least 
0A99n w.h.p. 

The next theorem contains the statements about the marginals of the truth values of individual variables. 

Theorem B.3 There exist constants ko , po > such that for all k > ko, and po < p < k In 2 — 2 In k the following 
two statements hold. 

I.lf9> . f 2 (1 + l/p 2 + k/2 k ~ 2 ), then w.h.p. for at least 9n/3 variables x G Vt we have 

M a (* t ) £ [0.01,0.99]. 

2. Ifln(n)/n < 9 < p(l — l/ j o 2 )/(fcln2), then w.h.p. for all but exp(—p)9n variables x £ Vt we have 



Mx(*t) G 



0, 2" fc/2 



U 



1 - 2" fe/2 ,l 



Theorems 12 .2142 .5 1 follow directly from Theorems IB. 114B31 bv reordering the individual statements according 
to the phases they appear in, apart from part 4 of Theorem l2.41 whose proof is given in Appendix lF.2l After stating 
some preliminaries in Appendix ICl we will prove Theorem IB. II in Appendix ID] Then, in Appendix [E] we will 
prove Theorem IB. 21 Further, Appendix [F]contains the proof of Theorem IB. 3 1 Finally, in Appendix|G]we prove 
Theoreml2.7l 



C Preliminaries 

Recall that Vt = {xt+i, . . . ,x n }. In addition, we let L t = {x t +i, Xt+i, ■ ■ ■ ,x n , x n }. For a literal / let \l \ be the 
underlying variable. For a formula $ on V = {xi, . . . , x n }, an assignment a G {0, 1} , and 1 < t < n we let 
$t jCr denote the formula obtained by substituting a(x s ) for a; s for all 1 < s < t and simplifying. 

We need the following Chernoff bound on the tails of a binomially distributed random variable X with mean 
A: for any t > 

P(X > X + t) < cxp (-t-<p(t/X)) and P(X < X — t) < exp (— f • ip(-t/X)) . (11) 

where 

(p(x) = (1 + x) ln(l + x) - x. (12) 
We will need the following consequence of Theorem l5.2l (cf. Corollary 15. 3l in the main part of the paper). 

Corollary C.l Let 1 < t < n. Let ($ t , at) be a pair chosen from the experiment U1-U4. Then w.h.p. 

iln|5(* t )| > (l-V")ln2 + rln(l-2- fc )- J. (13) 

Proof. Let $ be a formula such that i In |<S($)| > In 2 + rln(l - 2~ k ) - kr/A k . By Theorem I5T21 the random 
formula $ has this property w.h.p. Thus, it suffices to show that for a random a <E <S($) the bound ( fT3l l holds 
w.h.p. To this end, let I = {0, 1}*. Moreover, for each a <G {0, 1}™ let a\t be the vector (cr(xi), . . . , <r(x t )) 6 X. 
For each it, el let Z(<7*) be the number of assignments a G <S($) such that a\ t = cr*. If cr G <S($) is chosen 
uniformly at random, then for any cr* G 1 we have 



P[o-| t = o-»] = Z(cr,)/Z, where Z = ^Z(r) = |5($)| 

rex 



Let £ > be a sufficiently small number and let 



q = P [Z(o\ t ) < exp(-t In 2 - £n) ■ Z) , 
where a G S(&) is chosen uniformly at random. Then 



<T,eX:Z(<r.)< 



9 = V Z{a,)/Z<% Z <cxp(-gn) 



cip({ti+tln2) 



whence the assertion follows. □ 

In Section|4]we introduced the experiment P1-P4, which led to the planed model Vk(n, m). In addition, we 
need the following variant of the planted model. 

PI'. Choose an assignment cr' € {0, 1} V uniformly at random. 

P2'. Choose a formula by including each of the (2 k — 1) (?) possible clauses that are satisfied under cr' with 
probability p = m/((2 k — 1)(?)) independently. 

P3'. Substitute cr'(xi) for Xi for 1 < i < t and simplify to obtain a formula <&J. 

P4'. The result is the pair (<&' t , cr' t ), where cr' t : V t -> {0, 1} , x H> er'(x). 

Steps Pl'-P2' of this experiment induce a probability distribution V' k (n, m) on formula/assignment pairs. The 
following corollary establishes a connection between this distribution and the distribution U k (n, m). 



Corollary C.2 ([ID Suppose that k > 4 and < r < 2 k In 2 — k. Let 8 be any property of formula/assignment 
pairs. //P^(„, m ) [£]>!- cxp(-fcrn/4 fc ) then P Uk (n, m ) [£] = 1 - o(l). 



We will need the following elementary observation about the distribution V' k (n, m). 
Lemma C.3 Let ($, cr) be a pair chosen from the distribution V' k (n, m). 

1. For each literal I that is true under a the number of clauses supported by I is binomially distributed Bin( — • 
(«),m/((2* -!)(£)))• 

2. For any integer D the number of literals I that support fewer than D clauses is binomially distributed with 
mean 



n ■ P 



, k fn\ m \ „ 



(14) 



Proof. Without loss of generality we may condition on a assigning the value true to all variables. For any variable 
x let S x be the set of all possible clauses in which x is the only positive literal. Then |5 X | = feftZi) = ' (&)■ 
(First choose one of the k slots where to place x, then choose the k — 1 other variables occurring in the clause; 
the signs are prescribed by x being the unique positive literal.) Moreover, let S x be the number of clauses from 
S x that actually appear in the random formula F. As each of the clauses in S x is included in F with probability 
p = m/((2 k — l)(fc)) independently, S x has a binomial distribution Bin(- • (?) ,p). This establishes 1. 

Since for any two variables x, y we have S x D S y = 0, the random variables S x are mutually independent for 
all variables x. Therefore, the number S = J2 X ^{s x <d} of variables supporting fewer than D clauses in F is 
binomially distributed as well. □ 

There is a natural way to associate a bipartite graph with a fc-CNF $, known as the factor graph. Its vertices 
are the variables and the clauses of $, and each clause is adjacent to all the variables it contains. For a variable x 
we let N-s(x) be the subgraph of that is spanned by all vertices at distance at most 3 from x. A variable x is tame 
if Nz(x) is acyclic and contains no more than ln(rt) variables. The following is a well-known fact about random 
fc-CNFs. 

Proposition C.4 Suppose that k > 3 and < r < 2 k In 2. W.h.p. all but o{n) variables are tame in <&. 



Finally, the following lemma expresses an elementary 'expansion property' of the random formula 



Lemma C.5 There is a number \ = x(^') > such that for all < r < 2 k the random formula has the 
following property w.h.p. 



There is no set Q of 1 < \Q\ < x n variables such that the number of 
clauses containing at least two variables from Q is at least 2\Q\. 



(15) 



Proof. We use a first moment argument. Let 1 < q < \n and let Qq = {xi, . . . , x q } be a fixed set of size q. For 
any set Q we let Y(Q) be the number of clauses containing at least two variables from Q. Moreover, let X q be 
the number of sets Q of size q such that Y(Q) > 2q. Since the distribution Fk(n, m) is symmetric with respect to 
permutations of the variables, we have 



EX q < 



P [Y{Q ) > 2q] < exp [q(l + \n(n/q))} ■ P [Y(Q ) > 2q] . 



(16) 



Furthermore, the probability that a random fc-clause contains two variables from Qq is at most ( k )(q/n) 2 (because 
for each of the (5) pairs of 'slots' in the clauses the probability that both of them are occupied by variables from 



Qo is at most (q/n) ). As Fk{n, m) consists of m independent fc-clauses, Y(Qq) is stochastically dominated by a 



binomial random variable Bin(m 
P [Y(Q ) > 2q 



< 



< 



exp 



< exp 




Consequently, assuming that q/n < x is sufficiently small, we get 

> 2q 



2q 



k )(q/n) 2 m 



1 



ek 2 r 



< 



exp 



-1.9q ■ In 



[by the Chernoff bound ( [Til l 

4 



2ofc 



ek 2 2 



(17) 



Choosing \ = x{k) sufficiently small, we can ensure that (q/n) 1 / 4 < x 1 ^ 4 < 4/(c/c 2 2 fc ). Plugging this bound 
into (1171) . we get 



P[Y{Q Q )>2q] < cxp[-l.lg-ln(n/q)] 

Combining ( TTBT ) and ( fl~8b , we get EX q < exp [-O.lqln (n/q)] . In effect, E^i 
Markov's inequality implies that w.h.p. 



<9<X" q 



0, in which case (fT3T l holds. 



<q<\n 9 



X q = 0(n 



-o.i 



(18) 

). Hence, 

□ 



D Proof of Theorem EE 

D.l Loose variables 

Let a be a satisfying assignment of a fc-CNF $. Remember that a literal I supports a clause C of $ if / is the only 
literal in C that is true under a. Moreover, we say that a literal I is 1-loose if it is true under a and supports no 
clause. In addition, I is 2-loose if / is true under a and each clause that / supports contains a 1-loose literal from 
L t . Thus, any 1-loose literal is 2-loose as well. The key step of the proof is to establish the following. 

Proposition D.l Suppose that 9 > 3 cxp(p)(\n p + 10)/fc and r < 2 fc ln2 — k. Let ($,cr) be a random pair 
chosen from the distribution Uk (n, m). Then there are at least 0.9999n 2-loose literals in L t w.h.p. 



To prove Proposition lD.il we start by estimating the number of 1-loose variables. 

Lemma D.2 Suppose that 8 > exp(p) /k and p < k In 2. Let ($, a) be a random pair chosen from the distribution 
Vt (n, m). With probability at least 1 — exp(— k2 2 ~ k n) the number of 1-loose in L t is at least On ■ exp(— p)/2. 

Proof. By Lemma lC3l the number X of 1-loose literals in L t has a binomial distribution with mean 





\ (k 


(I) 


EX = 6n-P 


Bin - • 




\ n 





= 



HI) 



On- 



(2* -1)0 



On ■ exp 



kr 



6>nexp(-p- p/(2 k - 1)). 



As 9 > exp(p) /fc and p < fc In 2, the Chernoff bound (fTTT i shows that for large enough fc 



P [X < 9nexp(-p)/2] < exp 



On 



8 exp(p) 



< exp(-fc2 2 - fe n) 



as desired. 



□ 



Lemma D.3 Suppose that 9 > 3 exp(p) (In p + 10)/fc, p > po with po as in Lemma \D.2\ and that fc is sufficiently 
large. Let (<3?, a) be a random pair chosen from the distribution V k (n, m). Then with probability at least 1 — 
exp(— k2 1 ~ k n) the number of 2-loose literals in L t is at least O.9990n. 

Proof. To simplify the notation, we are going to condition on a being the all-true assignment; this is without 
loss of generality. For each variable x G Vt we let S x be the number of clauses supported by x. Moreover, let 
S = J2x<EV t and let X be the number of variables x G Vt such that S x = 0. Thus, X equals the number of 
1-loose variables. 

Let £ be the event that X > 0nexp(— p)/2 and S < 2p0n. Since the number of possible clauses with 
precisely one positive literal in L t is S has a binomial distribution Bin[6'n(^~|) , m/((2 fc — 1) (?))]■ 

Therefore, Lemma lL\2l implies that 



P [-.£] < P [X < On exp(-p) /2] + P [S > 2p0n] 

f n-\ 



< cxp [-fc2 2 " fe n] + P 



Bin 9n 




> 2p9n 



(19) 



We have 



9n 



n-1 



(2*-l)G) 



< 



p9n. 



Hence, combining ( TT9b with the Chernoff bound (fTTT i. we obtain for sufficiently large fc 

P h£] < cxp [-fc2 2 ~ fe n] + cxp [-0.99p^n] < 2 cxp [-fc2 2 ~ fe n] 



(20) 



where in the last step we used the assumption that p > po for a fixed constant po > 0. 

Let us now condition on the event that S = s for some number s < 2pn, and on the event £ . In this conditional 
distribution for each of the s clauses supported by some variable in Vt the fc — 1 negative literals that the clause 
contains are independently uniformly distributed. Therefore, for each such clause the number of negative literals 
y whose underlying variable y is 1-loose is binomially distributed Bin(fc — l,X/n). Consequently, the number 
T of clauses supported by some variable in Vt in which no 1-loose variable occurs negatively has a binomial 
distribution with mean s ■ P [Bin(fc — 1, X/n) = 0]. Hence, 



E [T\£] 



< 



2p8n ■ P [Bin(fc - l,0exp(-p)/2) = 0] 



= 2p9n ■ (1 - 9cxp(-p)/2) k - 1 < 2p9nexp{-9exp(-p)k/3) < 2cxp(-10)6>r 



Thus, the Chernoff bound (fTTT i implies that for fc > fco large enough 

P [T > O.OO10n|£] < exp(-O.OO10n) < cxp [-fc2 2 ~ fc n] . 



(21) 



Finally, the assertion follows from (l20l and (|2TT i. □ 

Proof. [Proposition ID. II Let £ be the event that a pair (F,a) G A n rn has at least O.9990n 2-loose literals. 
Lemma ILxH shows that 



V' k (n,m) 



[£]>!- cxp(-fc2 1 " fc ?i) > 1 - exp(-fcrn/4 fc ). 



(22) 



Moreover, Corollary IC.2l and (|22> imply that P^ fc (n,m) [£\ = 1 — as desired. □ 
Proof. [Theorem lBTl part 1] By Fact l4. H it suffices to prove the desired statement for the experiment U1-U4. 
Thus, let ($, a) be a pair chosen from the distribution Uk (n, m). Without loss of generality we may condition on 
a being the all-true assignment. Let C be the set of all tame variables that are 2-loose. Then by Propositions IC. 4 
and ID. II we have C > (0.999 — o(l))0n > O.990n w.h.p. Assuming that this is the case, we are going to show 
that if x G C, then there is a satisfying assignment r such that r(x) ^ <r(x) and dist(r, a) < ln(n). 



Thus, fix a variable x G C. If x is 1-loose, then we can just set t(x) = 1 — a(x) = and r(y) = a(y) = 1 for 
all y ^ x to obtain a satisfying assignment with dist(r, a) — 1, because x does not support any clauses. Hence, 
assume that x is 2-loose but not 1-loose. Let C be the set of all clauses supported by x in ($, a). Any clause 
C G C contains a negative occurrence of a 1-loose variable xc £ Vt in C (by the very definition of 2-loose). 
Define t{x) = 0, t(xc) = for all C G C, and r(y) = er(z/) = 1 for all other variables y. 

We claim that r is a satisfying assignment. To see this, assume for contradiction that there is a clause U that is 
unsatisfied under r. Then U contains a variable from {x} U {xc ■ C G C} positively, while none of these variables 
occurs negatively in U. Hence, U $ C. Moreover, since the variables Xc, C G C, do not support any clauses, U 
indeed contains two variables from the set {x} U {xc ■ C G C} positively. There are two possible cases. 

Case 1: x occurs in U. Let C G C such that xc occurs in U as well. Then the factor graph contains the cycle 
x, C, xc, U, x, in contradiction to our assumption that x is tame. 

Case 2: x does not occur in U. There exist C\,C2 G C such that xc ± , xc 2 occur in C. Hence, the factor graph 
contains the cycle x,C\, xc\ , C, xc 2 , C2 , x, once more in contradiction to the assumption that x is tame. 

Hence, there is no clause U that is unsatisfied under r. Finally, since all the variable xc with C G C have distance 
two from x in the factor graph, and as x is tame, we have dist(er, r) < In n. □ 

D.2 Rigid variables 

The proof of the second part of Theorem IB. II follows the outline given in Section [6] Recall the function ip 
from ([12}. 

Proposition D.4 Suppose that k > 6 and < r < 2 k In 2 - k. Let \i = p ■ 2 k /(2 k - 1) and ( = (1 + p + 

p? /2) / exp(^), and assume that 2 k 6Qip{\) > p. Then w.h.p. in a random pair ($, a) chosen from the distribution 
Uk (n, m) no more than 2Q6n literals in L t support fewer than three clauses. 

Proof. Let S be the number of literals I G L t that support fewer than three clauses. We are going to show that 

Pv k (n, m) [S > 2(6n] < cxp(-krn/4 k ). (23) 

Then Corollary IC.2l implies the assertion. 

In the distribution V' k (n, m) the random variable S is binomially distributed with mean (1 + o(l))9(n by the 
second part of Lemma lC.3l Hence, the Chernoff bound (fill shows that 

P Pfc( „, m) [5 > 2(n] < exp(-(l + (l))^(lH. (24) 

By the assumptions on p and 9 we have 6Qtp[\) > p/2 k ; hence, d23l follows from (l24l . □ 

Remember that a set S G L t of literals t-self-contained if each literal I G S supports at least two clauses that 
contain literals from {xi , x\ , . . . , Xt , Xt } U <S U S only, where S is the set of all negations of literals in S. 

Proposition D.5 For any k > 3 there is a number \ = x(^) > such that for any < r < 2 k In 2 - k 
the following is true. Let ($, a) be a random pair chosen from the distribution Uk (n, m). Then w.h.p. for any 
t-self-contained set S all variables x G S U S are xn-rigid. 

Proof. Let ($, a) be a random pair chosen from the distribution Uk (n, m). Without loss of generality we may 
condition on a being the all-true assignment. By Lemma [C31 there is a number \ = x(k) > such that ( fT~5T > is 
satisfied w.h.p., and we are going to assume that this is the case. 

Let S be a self-contained set. Suppose that r is a satisfying assignment such that the set Q of all variables 
x G S U § such that t(x) ^ cr(x) is non-empty. For each variable x G Q there are two clauses C\(x), C%(x) 
that are supported by x in a and that consist of literals from S U S only (because S is self-contained). Since 
t is satisfying and t(x) ^ cr(x), both C±(x) and 62(2:) contain another variable from Q. Hence, there are at 
least 2\Q \ clauses that contain at least two variables from Q. Thus, (T5[ implies that \Q\ > x n , an d consequently 
dist(cr,r) > \Q\ > xn. □ 



Proposition D.6 Suppose that k > 4 and < r < 2 k In 2 — k, and that < 6 < 1. Sef 

p2 fc 1 + /i 2 /2 w^-i ^ M ■ (exp(A/i) - 1 - A/i) 

" = 2*=T' C = exp( M ) > A=1-(1-3^ C ) ,7= ( W ) cxp(Ai ) 

a«c/ /ef /i(i') = — a; lnx — (1 — a;) ln(l — x). If £ < 1/3 anc/ 

0(Cln( 7 ) + MC))+p/2 fc <O, (25) 

f/ie« a random pair ($, er) chosen from the distribution Uu (n, m) has one of the following properties w.h.p. 

a. More than 29(,n literals in L t that are true under a literals support fewer than three clauses. 

b. There is a t-self-contained set of size (1 — 3()9n. 

Proof. Let p = m/ ((2 k — 1) (?) ). Let ($, a) be chosen from the distribution Vk (n, m). We may condition on a 
being the all-true assignment, and on the event that at most 2C,9n literals amongst xt+i,xt+i, . . . ,x n ,x n that are 
true under a support fewer than three clauses (as otherwise a. occurs). In fact, fix a set Z of 2(9n variables and 
condition on the event £ that all variables that support at most two clauses lie in Z. For any variable x Z we 
let 5a; be the number of clauses supported by x. Then the first part of Lemma lC31 implies that S x has a binomial 
distribution Bin(fc(^~J) , p) conditioned on the outcome being at least three. As a consequence, for any j > 3 



P 

P[S x =j\£] = - 



Bm(k{ n k zl),p)=3 



Bin(fc (":*), p) < 3 



^ T-^rVT^TS + («p(-i/M)/n) • (26) 
j!exp(/i)(l - C) 



Let X C {xt+i, ■ ■ ■ , x n } \ Z be a set of C,9n variables. For each x G X we let T x be the number of clauses 
supported by x in which a variable from XUZ occurs negatively. In a random clause supported by x the variables 
underlying the k — 1 negative literals in that clause are distributed uniformly over V. Therefore, given £ the 
probability that such a clause contains at least one variable from XUZis 

1 - (1 - \X U Z\/n) k - 1 + o(l) = 1 - (1 - 36>C) fe_1 + o(l) ~ A. 

Hence, if we condition on both £ and S x = j, then the probability that T x > S x — 1 equals j ■ (A + o(l))- 7-1 . 
Thus, letting 7 = /i • (cxp(A/i) — 1 — A/i) /(!—£) cxp(/i), we obtain from ( 



P[T X >S X -1\£] = ^P[T x >5 x -l|£and5 E =j]-P[5 K =j|£] 

J>3 

< (i + o(D) 'Tvf n ^- 

Given that £ occurs the events > 5 K — 1 are mutually independent for all x £ X. Therefore, 

P[VxeX :T X >S X -1\£] < ( 7 + o(l)) ec ". 

If b. does not occur, then there is a set X C V \ Z of size £n such that > S x — 1 for all x £ X. Hence, by the 
union bound the probability that b. does not occur is at most 

P[3X C V \Z, \X\ = (n : Vx € X : T x > S x - l\£] < ^ P [Vx e X : T x > S x - l\£] 

XCV\Z, \X\=9Cn 

< ( (1 ~ d fJ en ) ■ (7 + «(1)) 8C " < exp [(1 - 2C)MC/(1 ~ 20) • n] ■ (7 + (1)) C " 

< exp[6»n-((l-2C)/i(C/(l-2C)) + C-ln 7 + o(l))] <cxp(-pn/2 fe ) [by©], 



as desired. Finally, the assertion follows directly from Corollary IC. 21 □ 

Proof. [Theorem IETT1 part 2] Suppose that 1/k < 6 < exp(p)/(p 3 k). The goal is to verify (l25l l. Since 
h(() < C(l — In C)) proving d25l l reduces to showing 9Q [In 7 + 1 — In (] < —p/2 k , i.e., 

9(\n( C1 /()<-p/2 k . (27) 



Plugging in the definitions of 7 and £, we see that 



In 



(?) 


= In 







ep(e~xjp(\p) — A/i — 1) 
(l-0(l + M + M 2 /2) 



< In 



5e [exp(A/i) — A/i — 1] 



[for fx not too small] . 



Since 36*C < 4/(M for p > po sufficiently large, we have A = 1 - (1 - 36»C) fc_1 < 4fc6»C- Hence, 

4/i 2 exp(/i) 



A/x < ikp9( < 
Therefore, we obtain for p > p > po large 



exp(^) M 3 
/4c(A/i) 2 



<4/ M . 



< ln(64e//i 3 ) < -1. 



As 9C, > (/k > \p 2 cxp(-p) > \p 2 /2 k for k > k and p > p not too small, we thus obtain d27l ). 



□ 



D.3 Forced variables 

Let ($, cr) be a formula/assignment pair. A clause C forces a variable x £ V* if C contains k — 1 literals from 
{xi, xi, . . . , xt, Xt}, none of which satisfies C under a, and either the literal x or x, which does. 

Lemma D.7 Suppose that p > po, k > ko, and kO ~ ln(p) — 10. Then w.h.p. in a pair ($, a) chosen from the 
distribution Uk(n, m) at least 0.99K9ra variables in Vt are forced. 

Proof. Let T be the event that at least 0.991#n variables in V t are forced. We are going to show that 

Pv' k (n, m ) > 1 - cxp(-l.lp/2 fc ), (28) 

so that the assertion follows from Corollary IC. 21 

Thus, let ($', a') be a pair chosen from the distribution V' k (n, m). We may assume without loss of generality 
that a' is the all-true assignment. For each variable x e Vt the number of clauses that x supports has a binomial 
distribution with mean p — p-2 k / (2 k — 1). Furthermore, if C is a random clauses supported by x, then C contains 
k — 1 random negative literals; the probability that all of these are in V \ Vt equals (1 — 9 + o(l)) fc_1 . Hence, the 
number F x of forcing clauses for x is binomially distributed with mean 

e[f x ] = p(i - e + oil))*- 1 > P {i - of- 1 

> pcxp [-{9 + 9 2 ){k - 1)] > pexp [-6k - 9 2 k] > cxp(5). 

Therefore, for any x £ Vt we have P [F x = 0] < exp(— exp(5)), and the events ({F x — 0}) x£ v t are mutu- 
ally independent. Hence, the number Z of variables x £ Vt with F x = is binomially distributed with mean 
exp(— cxp(— 5))6n, and thus 

P [Z > 0.009<9n] < cxp(-0.0096»n) < exp(-l.lp/2 A; ) 

by Chernoff bounds. This proves (128V □ 

Proof. [Theorem iBTl part 3] To complete the proof of Theorem lB.il part 3, we need to deal with general 
values 1/n <C 9 < 9q = (ln(p) — 10)/fc. Let t = (1 — 9)n and to = (1 — 9o)n. To obtain a pair ($ t , a t ) from 
the distribution U1-U4, one can proceed as follows. First, choose a pair ($t , <7t ) from the distribution U1-U4 
with to variables decimated. Then, assign the variables in x £ Vt \ Vt with the truth values a ta (x), simplify the 
formula, and let at(y) — cr to (y) for all y £ Vt- We are going to use this experiment to analyze the number of 
forced variables in er t ). 

The above experiment shows that any variable x £ Vt that is forced in (4> to , cto) remains forced in at). 
Let J 7 be the set of forced variables in ($ to , cto). Given that \F\ = j, the set J 7 is a uniformly random subset of Vt . 
Hence, if we condition on the event that \F\ > 0. 99100^ then I^Ffl Vt \ has a hypergeometric distribution with 
mean at least 0.991#n. Therefore, by Chebyshev's inequality, we have \F Pi Vt\ > (0.9910 — o(l))n > 0.999n 
w.h.p. (here we use that 9n ^> 1). Thus, the theorem follows from Lemma lD/71 □ 



E Proof of Theorem E2 



E.l Shattering 

In this section we prove the first part of Theorem IB. 2 1 Consider a pair (&,a) chosen from the planted model 
Vk(n, m). Let $ t (T denote the formula obtained from $ by substituting the values cr(xi), . . . , <r(x t ) for the first t 
variables. Without loss of generality, we may assume that a = 1 is the all-true assignment. The main step of the 
proof is the summarized in the following proposition. 

Proposition E.l Let k > 6 and r > be fixed. Moreover, let < < 1 and let 

1 - (1 - a6) k 



V> : (0, 1) -» R, a m- -a01na - (1 - a)01n(l - a) +rln 1 

y 2 fc — l 

Suppose that there is a number a G (0, 1) such that 

ip(a) + p/2 k <0 and sup 0(a) < 01n2 + 2 fe pln(l - 2- fc )/fc - / o/2 fe . (29) 

0<a<a 

r/zen there is e = s(k, p) such that for <£> t generated by the experiment U1-U4, the set <S($t) is (a — e, a + e)- 



We will first show how Proposition IE. 1 1 implies the first part of Theorem IB. 2 1 The proof of Proposition IE. 1 
appears at the end of this section. To derive the first part of Theorem IB. 21 from Proposition IE. II we need to 
verify 



Lemma E.2 Assume that < < cxp(p — 2)/{pk). Let a = exp(2 — p). Then ip(a) < —aO/2. 
Proof. We have 

xj)(a) < a6(l -In a) - £ (l - (1 - a0) k ) < a0(l -In a) - £ (1 - exp(-afc0)) 
< a0(l - In a) - j- (ak6 - (afc0) 2 /2) = a6 [1 - In a - p(l - afc0/2)] , 

where we used exp(— z) < 1 — z + z 2 /2 for z > 0. Since kdp < cxp(p — 2) by assumption, our choice of a 
implies that ip(a) < a6 [1 — In a — p + a cxp(p — 2)/2] = —a9/2, as claimed. □ 



Lemma E.3 Assume that < < exp(p — 2)/(pk). Let a = cxp(2 — p). Then sup Q<a ip(ct) < 2c | fcp . 
Proof. Let < a < a. We have 

i/)(a) < 6(a-a\na-ap(l-ak6/2)). 
Let ipi (oi) be the expression on the r.h.s. Then 

d d 2 

—ipx (a) = [- In a - p + ak9] , — (a) = \k6 - 1 la] . 
da da 1 

Thus, our assumption on implies that -j^ipi(a) < for all < a < a, and therefore has a unique local 
maximum in the interval (0,a). To pinpoint this maximum, note that for ao = cxp(— p) the first derivative 
■4-ipi(ao) is positive. Moreover, at a\ = exp(l — p) we have ^V'l^i) < 0- Hence, the unique local maximum 
of ipi lies in the interval (ao, a\). To study the maximum value, consider the function -02 :a^a-«lna - ap. 
Its derivative is d/daip2(&) = P — m a, so that the maximum of this function occurs at ao- Furthermore, the 
quadratic term a M> a 2 k/2 is monotonically increasing in a. Therefore, 

sup ip(a) < sup ipi (a) = sup < 0(^2(010) + a^fc/2) = 30cxp(— p)/2. 

0<a<a 0<o<a Qo<ct<»l 



Finally, the assertion follows from the assumed bound on 0. 



□ 



Proof. [Theorem lB.2l part 2] Assume that p < k In 2 — In k and 

9 (l + p-' 2 + 2 2 - k ) <e<exp(p-2)/(pk). 



fcln2 

Let a = cxp(2 — p). Lemma lE~2l shows that 

«.)+,/* < ,/*_ _,,,,,<,/*_ 3^ 

Since p < fc In 2 — In the r.h.s. is negative. By Lemma IG31 we have 

6»ln2 + ^ln(l-2- fe )- p/2 k > e\sx2-?- - pl2 k - x 
k k 

> — +2 2 "Vln2 - p/2 k - 1 > — > sup -0(a). (30) 
Thus, the assertion follows from Proposition lE. II □ 
E.l.l Proof of Proposition IBTT1 

In the rest of this section we keep the notation and the assumptions from Proposition lE.il Let 

b = 01n2 + 2Vn(l - 2" fc )/fc - p/2 k . 

Lemma E.4 There exist numbers £ > 0, < a\ < ai < 1 5mc/z f/iaf a pair ($, a) chosen from the distribution 
Vh (n, to) /las the following two properties with probability at least 1 — exp(— (£ + p/2 k )n). 

1. <f>t,ir does not have a satisfying assignment r with ain < dist{a, r) < a^n. 

2. \{t e S($ t ,a) ■ dist(a, r) < a 2 n}\ < exp((fe - £)n). 

fVoo/ For a > we let X a = \{t E S tt<T (F) : dist(a, r) = adn}\ . Note that 

«■ <- (Di^-^T- 

Taking logarithms and bounding the binomial coefficient via Stirling's formula, we obtain 

< "0(a)- (32) 



Let a £ (0, 1) be such that 0(a) + p/2 k < (cf. d29l)). As is continuous there exist 0<ai<a<a,2<l 
and £i > such that 

sup ip(a) < -p/2 k - 2£i. (33) 

ai <a<a2 

Combining (f32t and (l33l . we conclude that EX Q < exp [— n(p/2 k + 2£o)l for all ai < a < a2- Summing over 
integers ain < j < a^n, we see that for large n 

J2 EX i/n , 6 < n cxp [-n(/>/2 fc + 2f„)] < cxp [-n(p/2 fe + &)] . 

ain<j<a2n 

Hence, by Markov's inequality the probability that there is a satisfying assignment r that coincides with a on 
the first £ variables such that ayn < dist(er, r) < ain is bounded by exp(— n(p/2 k + £o))- This proves the first 
assertion. 

Since we are assuming that sup 0<Q , <a "0(a) < b — p/2 , and as d33l shows that 0(a) < —p/2 k — 2£i < 
— /o/2 fc — 2£i for all a < a < a2, there is a number £2 > such that 

sup 0(a) < b-p/2 k -3£ 2 - 

0<a<a 2 



Hence, ( f32b implies that 

F,X a < exp(nip(a)) < exp(n(6 - p/2 k - 3£ 2 )) for all < a < a 2 . 
Taking the sum over integers < j < a 2 n, we get for large enough n 

VX 3/n < nexp(n(& - p/2 k - 36)) < cxp(n(6 - p/2 k - 2&)). 

0<j<a2n 

That is, the expected number of assignments t G <S($t ](7 ) such that dist(er, r) < <22n is bounded by cxp(n(6 — 
/9/2 fc — 2^2))- Hence, Markov's inequality entails that with probability at least 1 — cxp(— n(p/2 k + £2)) there are 
at most cxp(n(& — £2)) such satisfying assignments r. This proves the second assertion. □ 

Corollary E.5 There exist numbers £ > 0, < a\ < a 2 < 1 such that a pair ($, a) chosen from the distribution 
Uk (n, m) enjoys the two properties stated in Lemma \E.4\ with probability at least 1 — exp(— £n). 

Proof. This follows directly from Lemma |E~41 and Corollary 14. 21 □ 

Proof. [Proposition lE. II Let £, 01, a 2 be the numbers provided by Corollary IE. 5 l and let (<£>, a) be a pair chosen 
from the distribution Uk(n, m). With each assignment r G iS(<£> t cr ) we associate a set 

C(r) = {x£ S(*t,«r) : dist( X , r) < am}. 

Moreover, we call r G <S($t i<T ) good if \C(t)\ < exp((& — £)n) and there is no \ G <S($ ti(r ) such that ain < 
dist(x, t) < a 2 n. Let S goo d be the set of all good r G S(<& t ,a) and <Sf, a£ 2 = 5($ ti<7 ) \ S goo d- Corollary IE. 5 1 and 
our choice of b ensure that F has the following two properties w.h.p.: 

|5($ t , CT )| > 2*exp(6n), (34) 
\S good \ > (l-exp(-fn))-|5(* t , ff )|. (35) 

Assuming that ( 1341 and d35l l hold and that n is sufficiently large, we are going to construct a decomposition of 
S(&t,a) into subsets as required by SH1-SH2. To this end, choose some o~\ G S goo d- Having defined a%, . . . , crj, 
we choose an arbitrary 07+1 G <Sgood \Uj=i ^( <T j)' unless this set is empty, in which case we stop. Let <ti, . . . , ctjv 
be the resulting sequence and define 

Ri =C{ai)\ \JC{a s ) foil <1<N, W dRo=S($ tia )\\jR l . 

3=1 1=1 

Then <S($ 4jIT ) = i? U • • • U R N . (Observe that possibly R Q = while i?; ^ for all 1 < I < N as 07 G 
Furthermore, for each 1 < I < N we have i?; C £(07) and thus 

< [C(o"z) I < exp((o — £)n) [because 07 is good] 

< |S($ ti<T )|-exp(-£ra) [by®]. (36) 
Furthermore, as R C <S& a d, ( T35T l implies 

|i?o| < |5 6od | < exp(-£n)) • |S($t, CT )| . (37) 

Combining (TSoT l and d37l i we see that the decomposition i?o, • ■ • , Rn satisfies SHI. Furthermore, SH2 is satisfied 
by construction. □ 

E.2 Condensation 

Here we prove the second part of Theorem IB. 21 The following proposition reduces that task to a problem in 
calculus. 



Proposition E.6 Let k > 3 and r > be fixed. Let < < 1 a/it/ /ef 



-0 : (0, 1) -> R, a (->■ -afllna - (1 - a)01n(l - a) +rln 1 - 



1 - (1 - a6 

2 k - 1 



If there is a number a E (0, 1) such that 



sup 

a<a<l 



(38) 



f/ie« #4 « 2a9-condensed. 

Proof. Let ($, tr) be a pair chosen from the planted distribution Vk(n, m). For a > we let 

I a = |{r £ <S(*-t. a) : dist(cr, r) = a^n}| . 



Then EX„ < 



6n 



1 



l-(l-ae)* ; 



and taking logarithms we obtain -lnEX a < ip(a). Hence, — lnEXc < 



—p/2 k for a > a by d38l . Thus, by Markov's inequality we have 

P [3t € S t (<Z>t,<r) :d{a,T) > adn] < On ■ exp(-(0(l) + p/2 k )n) < cxp(-pn/2 k ). 
Therefore, the assertion follows from Corollary [ 



□ 



Lemma E.7 Suppose that p < fcln2 — 21nfc and = (1 — l/p 2 ) fcl ^ 2 . Moreover, assume that p > po and 
k > kofor certain constants po 7 /cq. Let a = cxp(2 — p). Then rti^D is satisfied. 



Proof. Let h (•) be the entropy function. We have 



< Oh(a) - y(l - exp(-akO)). 
k 



To bound the r.h.s., we are going to consider several cases. 
Case 1: a < l/(kp0). As a > a = cxp(2 — p), we obtain 

ip(a) < aO [1 - In a - p + akpO/2] < aO 



akpO 



- 1 



< -aO/2. 



The assumption p < k In 2 — 2 In k ensures that the last term is smaller than —p/2 k . 
Case 2: l/(kp0) <a< l/(k0). We have 

ip(a) < aO [1 - In a - p + akpO/2] 

akpO 



< aO 



1 + \n(kp0) - p 



< aO [1 + \n{kp0) - p/2] 

< a0[l -lnln2 + 21np-p/2] 

< -aOp/A. 



[as a < 1/00)] 



The assumption p < k ln2 — 2 In k ensures that the last term is smaller than —p/2 k . 
Case 3: l/(kO) < a < a = 0.15. We have 

Ma) < Oh(a)-^{l-exp{-akO))<Oh{a Q )-^-{l-l/e) 
k k 



< 



h(a ) 



In 2 



1 + 1/e 



The choice of a ensures that the last term is smaller than — p/2 k . 



Case 4: a < a. As k6 = (1 - l/p 2 )p/ In 2, we get 

i>(a) < 6h(a) - £(1 - exp(-afc0)) < 6> In 2 - j(l - exp(-a (l - l/p 2 )p/ln2)) 

fx, rC 

< - [cxp(-a p) - I/P 2 ] • 
The last term is smaller than —p/2 k . 

□ 

Proof. rTheoremlR2l part 2] Let O = (1 - l/p 2 )p/(fcln2) and t = (1 - 6 )n. Suppose that 9 > 9 Q . Then 
<l> t is obtained from <t> to by assigning some further variables. Therefore, 

max r) : er, r e < max{d(cr, r) : er, r e 5(# to )} . 

Hence, Proposition lE.6l and Lemma IET71 imply that €> t is cxp(2 — p)-condensed w.h.p. □ 

E.3 Pairwise distances 

Recall that Q t .a denotes the formula obtained by substituting the values a{xi) for Xi for 1 < i < t. 

Lemma E.8 Suppose that 9 > -^(1 + l/p 2 + k/2 k - 2 ). Let ($,er) be a pair chosen from the distribution 
Uk(n,m). W.h.p. we have 

\{t e S(<$>t,v) ■ dist(r,a t ) < O.490n}| < exp(-fi(n)) \S t (<f>)\ . 
Proof. We need to work with the function 

^(a) = -a6 In a - (1 - a)6 ln(l - a) + ^/ In 1 1 1 ~ (1 ~ 



k V 2 fe - 1 

By Corollarv lC.il w.h.p. — In |5($ t )| > 9 In 2 + r In (l — 2~ fc ) — p/2 k . From now on, we are going to work with 
the planted model V' k (n,m). We are going to show that 



sup ip(a) - 9 In 2 - r In (l - 2~ k ) < -p/2 

Q<0.1 



fc-1 



Then the assertion follows from Corollary IC.2I We have 

9 k n 

1 



tf(a)-*]n2-^ln(l-2-*) = *(M«) -ln2) + ^ . d -^(l - (1 -«*)*) 



fe v 7 ' v ' ' k 

< 0(h(a) - In 2) + - (1 - atf)* + 2~ fc 

< 6>(/i(a)-ln2) + ^cxp(-a/c(9) + 2- fe . 

The differential of the last expression with respect to 9 is negative, and thus the function is monotonically decreas- 
ing in 9. Therefore, it suffices to consider the minimum value 9 = p/(k In 2). Thus, we obtain 

^(a)-01n2-^ln(l-2- fc ) < £ ( ^ - 1 + exp(-ap/ln2) > ) + 2~ k . 

k k \ in 2 / 

We consider a few different cases. 

Case 0: a < cxp(2 — p). Lemma lE31 shows that -0(a) < l/(kp) and ( f30b shows that 

6> In 2 + 2 fc - ln(l - 2~ fc ) > 6»ln2 - p/fc - p/2 fe . 
re 



Hence, 



V-(a)-6»ln2-^ln(l-2- fe ) < - (9 In 2 + f + p/2 k . 



k kp k 

Since we are assuming that 9 > , f „ (1 + l/p 2 + k/2 k ~ 2 ), the r.h.s. is smaller than p/2 k ~ 1 . 



Case 1: exp(2 — p) < a < exp(— p/2). Bounding the exponential by a quadratic function, we get 



?/>(a)-(91n2-rln(l-2- fe ) < 



< 



ap 
/cm 2 

ap 
k\n2 



1 — In a — p 



ap 



-1 



{apf 



2 In 2 



41n2 



2~ k < -p/2 k -\ 



provided that po < p < k In 2 — 2 In k. 
Case 2: cxp(— p/2) < a < 1/ (2p). Bounding the exponential by a quadratic function, we get 



V>(o0-6>ln2-rln(l-2- fc ) < 



ap 
/cm 2 



1 — In a — p ■ 



(ap) 2 
21n2 



2- k < -p/2 



fc-i 



provided that po < p < k In 2 — 2 In k . 
Case 3: 1/(2/?) < a < 10\n(2)/p. Suppose that a = x ln(2)/p for some 1/2 < x < 10 In 2. Then 



^(a)-6»ln2-rln(l-2" fe ) < 



;ln2 



(1 — In x — In In 2 + In p) — 1 + exp(— .x) 



As a; remains bounded away from 0, the term exp(— x) — 1 is strictly negative. Thus, the entire expression 
is smaller than — p/2 k ~ 1 for p > po sufficiently large. 



Case 4: 101n(2)/p < a < 0.49. We have 

7/>(a)-6>ln2-rln(l-2- fc ) < { 
The r.h.s. is clearly smaller than —p/2 k ~ 1 . 



p fh(0.1) 
In 2 



l + cxp(-10) +2 



□ Lemma [E~8l directly implies the third part of Theorem 



F Proof of Theorem B.3 



El Bounding the marginals away from 0, 1 



Here we prove the first part of Theorem IB. 3 1 We may assume that 9 > p/(k In 2). The goal is to show that the 
marginals of a substantial fraction of the variables x t+ i, . . . , x n are bounded away from 0, 1. 

We set up an auxiliary graph Q whose vertices are all pairs (x,t) of variables x G Vt and assignments 
r G S(&t)- A pair (x, r) is connected by an edge with another pair (x, r') if t(x) = t'(x). (Thus, the graph 
consists of components (x, ■) with x G Vt.) Lemma IE78l implies the following. 

Corollary F.l Let $ 4 is the formula obtained through the experiment U1-U4. W.h.p. we have 2\E(Q)\ < 
0.511\S t (^)\ 2 9n. 

Proof. We count the number of edges from each assignment r. By Lemma [E~8l almost all assignments t are such 
that the 'overlap' with almost all other assignments r' is at most 0.51#n. For such assignments, the number of 
edges incident with {(r, x) : x G Vt} is bounded by (1 + o(l))0.516'n. □ 

To bound the marginals away from 0, 1, assume that indeed 2\E(Q)\ < 0.511 I^S^^-) l 2 ^. Any variable 
x whose marginal does not lie in (0.01,0.99) is such that the set {(r, x) : r G iS^t)} induces at least (1 + 
o(l))0.99S($ t ) 2 /2 edges. Hence, if we let v be the number of such variables, then (1 + o(l))0.995($ t ) 2 ^ < 
2\E(G)\ < 0.511|5 t ($)| 2 6»n. Hence, v < °- 51 1 + o(1) gn < §0n. 



F.2 Concentration of the marginals about 0, 1 

To prove the second part of Theorem IB. 3 1 we need the following lemma. 



Lemma F.2 Suppose that 9 < p/(k In 2). Let ($, a) be a pair chosen from the distribution Uk (n, m). W.h.p. 
there is no set of variables Z C Vt of size 2kn/2 k < \Z\ < (ep)~ A 9n such that each variable in Z supports two 
clauses under a, each of which contains an occurrence of a variable in Z that evaluates to false ' under a. 

Proof. We work with the planted model VL (n, m). Let p be such that the expected number of clauses is m, i.e., 
(2 k — 1) (fc)p = m. Then the probability that a given set Z of size z is 'bad' is bounded by 

n 



-z). 



z [ k " 2 jPj < (ak9p) 2z 1 with a = z/(6n). 
Thus, the probability that there is a bad set of size z is bounded by 

^ (ak9p) 2z < (ak6p) 2z = [ca(k9p) 2 ] z < {cap 4 /ln 2 2) z < exp(- 



The assumption on z ensures that this is sufficiently small to move from the planted model to Uk (n, m) via 
Corollary E2 D 
Proof. [Theorem lB~3l part 2] If k9 < ln(p)/2, then the existence of forced variables immediately implies 
part 2 of Theorem IB. 3 1 Thus, let us assume that hi(p)/2 < k9 < pj In 2. Let ($, a) be a pair chosen from the 
distribution Uk(n, m). Let S be the set of rigid variables; by Theorem lB.il we have \S\ > p 3 exp(—p)9n w.h.p. 
Define an auxiliary bipartite graph as follows. The vertices of the graph are the variables in S and the satisfying 
assignments S(3>t,a)- Each variable x G S is connected with all r G <S($ tj0 .) such that t(x) ^ a{x). By the 
Lemma |R21 and because $ is exp(2 — p)-condensed (part 2 of Theorem lB.2K there is —> such that the number 
of edges of this bipartite graph is bounded by k2 1 ~ k n \St ($) | ■ Hence, the degree sum over the variables satisfies 



x£S 



d(x) < k2 2 - k n\S t {<S>) 



We may assume without loss of generality that a = 1 is the all-true assignment. Then the marginal p(x) equals 
1 — d{x)/ |<S t ($)|. Hence, the above bound on the degree sum shows that p{x) < 2~ k ^ 2 for all but 0.01#n 
variables x £ S. □ 



Proof. [Theorem l2~4l part 4] This follows directly by applying Lemma lR2l to the self-contained set obtained in 
Appendix |E21 □ 



G Belief propagation 



The proof of Theorem l2.7l is based on results from [8J. These results show that, in order to obtain Theorem 12.71 
we essentially have to verify that the outcome <fr t of the experiment U1-U4 enjoys certain quasi-randomness 
properties. We begin by stating the necessary properties. To this end, we define 

S t = exp(-c0/c), (39) 

where c > is a small absolute constant (independent of k, r, t, n). 

Fix a fc-CNF $ and an assignment a G {0, 1} V . Let <3?t )(T denote the CNF obtained from $ by substituting 
<t(xi), . . . , cr(xfc) for xi, . . . , x t and simplifying. Let G = G($>t,id,i) denote the factor graph. For a variable 
x G Vt and a set Q C Vt let 

N<x{x,Q) = {b e N(x) : \N(x) HQ \{x}\ <1 AO. 19k <\N(b)\ < 109k}. (40) 

Thus, N<i(x, Q) is the set of all clauses that contain x (which may or may not be in Q) and at most one other 
variable from Q. In addition, there is a condition on the length |iV(&)| of the clause b in the decimated formula 
$t i(T . Observe that having assigned the first t variables, we should 'expect' the average clause length to be 9k. For 
a linear map A : R v * — > R y * let ||A|| D signify the norm 

II A II l|ACI| l 
A n = max -—7. — . 

C eR^\{0} IICIL 



Definition G.l Let 8 > 0. We say that ($, a) is (S, t)-quasirandom if § satisfies QO and $ t cr satisfies Q1-Q4 
below. 

QO. There are no more than In In n redundant clauses. Moreover, no variable occurs in more than In n clauses of 
$. 

Ql. No more than 10~ 5 SOn variables occur in clauses of length less than 6k j 10 or greater than 106k. Moreover, 
there are at most lQ~ A 86n variables ieV t such that 

(^•E fe6 iV(x)2- |JV(6)l >l. 

Q2. IfQcVt has size \Q\ < 86n, then there are no more than 10~ 4 <5#n variables x such that either 

J2 ^ lN{b)l > P(0k) 5 8, or (41) 

b£N(x):\N(b)nQ\{x}\ = l 

2 |JV(6)nQ\{ 3: }|-|JV(6)| > A j or (42) 
beN(x)-.\N(b)r\Q\{x}\>i 



E 



sign(x, b) 



2 WO I 
b67V<!(a;,Q) 



> looo- (43) 



Q3. For any 0.01 < z < 1 and any set Q CV t of size 0.01<5(n - t) < \Q\ < lQQ5(n - t) we have 

]T \N(b)\ < 1.01|Q|/z. 

6:|AT(6)nQ|>z|iV(6)l 
Q4. For any set Q C 14 of size \Q\ < 10<5(?i — t) the linear operator 

A Q : R v * -+ R v \ Yl E 2_|Ar(b) ' • si S n ( x > & ) si § n (^ & ) r ^) (44) 

^6eiV< 1 (a;,Q)3/eJV(6)\{a} / x£Vi 

has norm ||AQ|j n < 5 A 6n. 
With respect to Q0, we have 

Lemma G.2 (IB)) The random formula <& satisfies condition Q0 w.h.p.,for any density < r = m/n < 2 fe In 2. 

Let $ be a fc-CNF and let 6 > 0. For a number 5 > and an index I > t we say that a;; is (5, t)-biased if the 
result ^ w) of the BP computation on $ t cT differs from i by more than 5, i.e., 

K($t, CT ,o/)-l/2| >5. 

Moreover, ($, <r) is (<5, t)-balanced\f no more than (56>n variables are (6, t)-biased. 

Theorem G.3 ([8]) There is po > such that for any k, r satisfying po ■ 2 k /k < r < 2 k In 2 and n sufficiently 
large the following is true. Suppose ($,(7) is {8 t ,t)-quasirandom for some I < t < T = (I — ln(p) / (c 2 k))n. 
Then is (St,t)-balanced. 

At the end of this section, we will verify that random formulas chosen from the distribution V' k {n, m) are indeed 
quasirandom. 

Proposition G.4 There exists a constant po > such that for any k, r satisfying po ■ 2 k / k < r < 2 k In 2 there is 
£ = £(fc, r) > so that for n large and S t , T as in Theorem \G.3\ the following is true. Let ($, er) be a pair chosen 
from the planted model V k (n, m), given that a ~ 1 is the all-true assignment. Then 

P [($,0-) is (S t ,t)-quasirandom\Q0] > 1 — cxp [— p2 1 ~ k n\ 

for any 1 < t < T. 

Finally, Theorem l2.7l follows by combining Corollary 14.21 Theorem lG.3l and Proposition lG.4l 



G.0.1 Proof of Proposition[Gl4l 



Let <&' = $>' k (n,m) be a random formula obtained by including each possible clause with probability p = 
m /(2 fc (™)) independently. 



Proposition G.5 (|8, Appendix E]) There exists a constant po > such that for any k, r satisfying po ■ 2 k jk < 
r <2 k In 2 for n 
at least 1 — exp 



r < 2 k In 2 for n large and 8t, T as in ( 1591 ) the following properties hold for a random formula €>' with probability 



-10 J2 s <t ^ s f or an y 1 — * — T, given that 3?' satisfies Q0. 

1. Ql and Q3 are satisfied. 

2. For any set Q of size \Q\ < <5#n f/iere are af mosf 10 _5 <5#?i variables x that satisfy either ( 1471 ). ( 1421 ). or 



sign(x, 6) 



bG-/V<i(z,Q) 



> 2W (45) 



3. For any Q the operator Aq from \44\ satisfies ||Aq|| d < <5 4 (n — t)/2 

Let $ t be the formula obtain from $ by substituting the value 'true' for x\,..., Xt-\ and simplifying. Since 
the 5 S form a geometric sequence, we have 

St = J> 

Observe that 



ck exp(cdk) 



6Sn > 10 15 S t 

if P > Po is chosen sufficiently large. 

Lemma G.6 There exists a constant po > rac/i that for any k, r satisfying po ■ 2 k /k < r < 2 k In 2 the following 
is true for the random formula <!>' w/f/j probability at least 1 — exp(— p2 2 ~ k n). 

1. The total number of all-negative clauses is bounded by 2 1 ~ k m. 

2. For each variable x G Vt let N x be the number of all-negative clauses in which x appears. Then the number 
of variables x G V t with N x > 2°- oiek is bounded by 5 2 9n. 

Proof. The first assertion simply follows from Chernoff bounds. With respect to the second assertion, assume 
that the first claim occurs, i.e., the total number of all-negative clauses is bounded by 2 1 ~ k m = 2pn/k. Then 
for each variable the average number of occurrences in such clauses is bounded by 2p. Therefore, the total 
number of variables that occur more than 2 0iefe times is bounded by 2p ■ 2~ n. By symmetry, the number 
of such variables that are amongst the last 9n variables is (asymptotically) binomially distribution with mean 
2p ■ 2~ omek 9n. Therefore, the second assertion follows from Chernoff bounds. □ 

Proof. [Proposition lG.4l Let ($, a) be a random pair chosen from the distribution V k (n, m). We may assume 
without loss of generality that a is the all-true assignment. Thus, the formula $ is obtained by including each 
clause that does not consist of negative literals only with probability p = mj ((2 k — !)(?)) independently. Now, 
let <£>' be the formula obtained by addition to $ each of the (™) all-negative clauses independently with probability 

p. Then has distribution &' k (n, m ■ 2 2 _ 1 ). Thus, with probability at least 1 — exp[— 10 2^2 s <t trie f° rmu l a 
$' has the properties 1.-3. from Proposition |G3] Let us condition on this event. 

Since <£' contains $ as a sub-formula, the fact that $' enjoys properties Ql and Q3 implies that the same 
is true of $. Furthermore, any variable x for which either (HTt or (l42t is true in $ has the same property in $' 
(because the expressions on the left hand side are monotone with respect to the addition of clauses). With respect 
to the expression in d43l . we decompose the sum for the pair ($, a) as 

S x ($,a)=S x (&,a)-R x , 

where R x sums over all clauses that are in $' but not in $. Due to Ql, we may assume that only clauses of length 
at least 0.19k occur in the sum R x . Thus, letting N x denote the number of clauses in $' \ $ containing x, we get 



\Rx | < 2~ 0A9k N x . The second part of Lemma IGT61 implies that for all but S 2 9n variables we have N x < 2 am9k . 
Hence, R x is tiny for all but 5 2 6n variables. This shows that $ satisfies Q2. 

With respect to Q4, let D be the difference of the two linear operators for $ and Only clauses of length 
at least 0.19k and at most 108k contribute to D. Hence, letting N denote the number of all-negative clauses, we 
have 



\\D\\u < 2 



-0.16k 



{lOOkfN. 



Since N < 2 



2pn/k by Lemma lCT6l we thus get 



|£>|| D < 2009n(6k)2 



Hence, the third part of Proposition lG.5l implies that $ satisfies Q4. 



□ 



